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Abstract—This study aimed to compare 5th graders’ 
scores obtained from Scratch projects developed in the 
framework of Information Technologies and Software 
classes via Dr Scratch web tool with the scores obtained 
from Computational Thinking Levels Scale and to 
examine this comparison in terms of different variables. 
Correlational research model was utilized in the study 
that 31 students participated in. Students were taught 
basic programming by using Scratch during a 6-week 
period. At the end of training, students’ programming 
skills were measured via Dr. Scratch web tool. 
Computational thinking skills were measured using 
Computational Thinking Levels Scale which includes 5 
factors: creativity, problem solving, algorithmic thinking, 
collaboration and critical thinking. Data were analyzed 
for internal reliability to calculate scale reliability. 
Cronbach Alpha reliability coefficient was found to be 
0.809. It was found that scores obtained by students by 
using any of the measurement tools did not differ 
according to gender or period of computer use, however, 
a high level significant relationship was observed 
between students’ programming skills with Scratch and 
their computational thinking skills. 


Index Terms—Scratch, Thinking Skills, Computational 
Thinking, Dr. Scratch 


I. INTRODUCTION 


It can be argued that advanced technology has 
changed the skills that the individuals are expected to 
have such as computational thinking skills. It is believed 
that computational thinking skills will be necessary for 
everyone in mid-21" century [1]. It can be claimed 
computational thinking skills are basic skills that need to 
be acquired by all and not just by computer experts [2,3] 
Teaching these skills especially to children is a topic of 
discussion in educational systems [4]. It is known that 
many countries have recently attempted to include 
computational thinking skills in their curriculums [5]. 
Computational thinking seeks to solve problems, designs 
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systems and comprehends human behaviors with the help 
of basic computer sciences [1]. Computational thinking is 
an important skill to educate individuals to identify 
problems, have a command of problem solving process 
and solve problems more _ productively [6]. 
Computational thinking aims to highlight skills such as 
creativity, logical thinking and critical thinking to 
develop problem solving skills with the help of 
computers [7]. International Society for Technology in 
Education (ISTE) [8] states that computational thinking 
cannot be fully defined without sub skills such as creative 
thinking, algorithmic thinking, critical thinking, 
cooperative learning and communication _ skills. 
Definitions of computational thinking skills point to the 
importance of training in programming to acquire sub 
skills [9-11]. Hence, it can be argued that training is 
significant to teach computational thinking skills [12]. 

For beginners, programming seems to be a hard task 
which can only be managed by individuals with high 
level training [13, 14]. This belief may be related to the 
fact that programming involves the use of many skills in 
combination such as generalization, abstraction and 
critical thinking [15]. In order to facilitate training, it is 
suggested to use programming tools and methods such as 
cooperative learning, game based learning, project based 
learning, simulation and drag and drop [14]. Instead of 
dealing with the complex structure of traditional 
programming languages, primary school _ students’ 
computational thinking skills can be measured via 
programming tools such as Scratch [12]. There are 
programming tools like Scratch, Alice, Microsoft Small 
Basic and Toontalk that emphasize visual aspects such as 
drag and drop. It is suggested that beginners use these 
programming tools [12,16] . Since they are free and have 
easy-to-use interfaces, it is thought that these 
programming tools provide significant advantages for 
beginners and primary school students. 

Among these vial programming tools, Scratch is 
distinguished due to its interface, language support and 
its feature of sharing projects [10]. It is known that 
Scratch is a program for beginners in programming, logic 
and algorithmic thinking skills [17 — 19]. 
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2 Comparing Students’ Scratch Skills with Their Computational Thinking Skills in Terms of Different Variables 


One of the most important problems facing educators 
in using Scratch support in lesson planning is related to 
tools developed to assess projects prepared with Scratch 
[20]. Dr. Scratch is a web tool that is used to score 
projects by entering source files or URL addresses of 
Scratch projects. Dr Scratch has been developed to 
analyze projects generated by primary and secondary 
school students using Scratch programming tool and to 
provide feedback regarding the development of 
computational thinking skills [20]. Dr. Scratch offers web 
based analysis and supports teachers and students in 
calculating the computational thinking scores of Scratch 
projects in 7 headings: abstraction, synchronization, 
analogy/paralellism, information, user interactivity, 
digital logic and control flow [20]. Dr. Scratch assigns 
three points to each heading cited above and the highest 
score that can be obtained is 21. In addition to supporting 
teachers in the assessment of Scratch projects, Dr. 
Scratch also assists students who want to develop 
programming skills on their own [20]. 

Literature includes many studies focusing on the 
contribution of Scratch visual programming tool to 
programming education [19, 21-27]. There is also 
research that focus on the projects developed with the 
help of Scratch visual programming tool in terms of 
computational thinking skills [20,28,29]. The current 
study differs from other studies since it asesses students’ 
computational thinking skill scores with two different 
measurement tools and compares them. This study aimed 
to compare the scores obtained by 5" graders from 
Scratch projects developed in the framework of 
Information Technologies and Software classes via Dr 
Scratch web tool with the scores obtained from 
Computational Thinking Levels Scale and to examine 
this finding in terms of different variables. Research 
problems are provided below: 


Research Questions 


a) Do Dr. Scratch scores for student projects differ 
according to gender? 

b) Do Dr. Scratch scores for student projects differ 
according to period of computer use? 

c) Do students’ Computational Thinking Levels 
scores differ according to gender? 

d) Do students’ Computational Thinking Levels 
scores differ according to period of computer use? 

e) Are there significant relationships between Dr. 
Scratch scores for student projects and _ their 
Computational Thinking Levels scores? 


II. METHODS 


A. Study Design 


The study utilized correlational research model. 
Correlation studies examine the relationship between two 
or more variables without any intervention [30]. This 
study aimed to investigate the relationship between the 
achievement scores obtained by Dr. Scratch web tool and 
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skill scores obtained from Computational Thinking 
Levels Scale. 


B. Study Group 


5" graders in Kastamonu province participated in the 
study. A total of 31 students took part in the research. 


Table 1. Gender Distribution 


Gender 
Total 
pice % sy % N 
Total 16 51,6 15 48,4 31 


16 of the participating students were females and 15 
were males. Students were taught Information 
Technologies and Software classes with Scratch support 
for 6 weeks and the projects generated at the end of the 
class were assessed. 


C. Teaching Implementation/Treatment 


Study group students were provided with Scratch 
programming tool training for 6 weeks in the framework 
of 2-hours a week Information Technologies and 
Software classes. Scratch programming tool user 
interface was presented to students in the first week of the 
study and samples of projects generated with the help of 
Scratch were displayed. Students were provided with 
information in the upcoming weeks about the use of 
Scratch. Code blocks were used to show how drag and 
drop process worked. They were shown how to register 
projects and how to upload registered projects to internet. 
Scratch web site explains how Scratch projects are 
developed. The use of blocks used in coding in Scratch 
program and how they are used in projects were 
presented with examples. Students were provided with 
opportunities to practice at the end of presentation during 
each class hour. Students were asked to generate 
authentic projects during the last two weeks of the 
process. The teacher acted only as a guide during project 
development phase. 


D. Data Collection Tools 


Computational Thinking Levels Scale: Computational 
Thinking Levels Scale, adapted to secondary school 
students, was developed by Korkmaz, Cakir and Ozden 
[2] to describe secondary school students’ Computational 
Thinking Levels. Factor, reliability and validity analyses 
were undertaken for the measurement tool. The scale is a 
5-pont Likert type scale with 22 items and five factors: 
creativity, problem solving, algorithmic thinking, 
cooperation/collaboration and critical thinking. The result 
of confirmatory factor analysis which was undertaken by 
using maximum likelihood technique showed item 
regression values between 0.507 and 0.872. Item-test 
correlation coefficients were between 0.655 and 0.862. 
Internal reliability analysis of the data was conducted to 
calculate the reliability of computational thinking scale 
and Cronbach Alpha reliability coefficient was found to 
be 0.809. 
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Dr. Scratch Web Tool: Students’ programming skills in 
Scratch were measured with the help of Dr. Scratch web 
tool. This tool was developed to calculate computational 
thinking scores by downloading the related Scratch 
project files from the computer or the Internet. 


Analyze your Scratch 
projects 


Fig.1. Dr. Scratch Web Tool Upload Screen 


Assessment on the web tool provides 3 points for each 
level and the highest score is 21. The web tool provides 
separate cores for each level. 


Level up 


& Flow control 

& Data representation 
@ Abstraction 

User interactivity 

& Synchronization 

© Parallelism 


Q Logic 


Fig.2. Dr. Scratch Score Levels 


The final score obtained from scoring identifies the 
projects to be in basic, developing or master levels. 


coe Se Score: oem Score: 17/21 c= 


nee 


The level of your project is... 
DEVELOPING! 


‘Youre doing a great job. Keep & upilt You're the master cf the universes! 


Fig.3. Dr. Scratch level screenshots 


Dr. Scratch web tool assesses Scratch projects under 7 
headings: flow control, data representation, abstraction, 
user interactivity, synchronization, analogy/parallelism 
and logic. Each heading provides 21 points by assigning 
1 point to each of the three features included in this 
heading. While scoring flow control group; programming 
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assigns one point when the feature is added to start the 
program, one point for the feature that allows the 
program to end instead of continuing forever and one 
point for programs that generates a cycle by eliminating 
code repetitions. While scoring data representation group, 
one point is assigned to identify the dimension and 
location of characters, one point for the program that 
involves new variables and one point for projects that 
creates and names lists. While scoring abstraction group, 
one point is assigned separately to group code blocks 
with different tasks, one point to design new blocks and 
one point to program and copy an object. While scoring 
user interactivity group, one point is assigned to starting 
the program with user interactivity instead of automatic 
start, one point is assigned to adding movement of 
characters in the program with user intervention and one 
point is assigned to ensuring sound and character 
interaction. While scoring synchronization group, one 
point is assigned to mutual interaction between two 
characters, one point is assigned to providing messages in 
character interaction and one point is assigned to 
uploading features such as changing stage. While scoring 
analogy/parallelism group, one point is assigned to work 
of more than one code group together, one point is 
assigned to assigning keys for parallel work and one 
point is assigned to forming conditions to generate events. 
While scoring logic group, one point is assigned to 
generate warning as a result of the created condition, one 
point is assigned to using more than one condition block 
and one point is assigned to using logical operation 
blocks. Addition of these points provides Dr. Scratch web 
tool score. 


E. Data Analysis 


SPSS 12.0 statistical program was used for data 
analysis. Spearman’s rho correlation test was utilized to 
investigate the relationship between the analysis scores 
obtained by students from Dr. Scratch web tool and the 
scores obtained Computational Thinking Levels Scale. 
Mann Whitney U test was used to examine the 
differences in Dr. Scratch web tool score and 
Computational Thinking Levels Scale achievement 
scores. 


III. FINDINGS 


A. Students’ Dr. Scratch web tool scores based on 
gender 


Mann Whitney U test was used to examine whether 
students’ Dr. Scratch web tool scores differed according 
to gender. Table 2 presents the data obtained from the test. 
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Table 2. Investigation of Students’ Dr. Scratch Web Tool Score According to Gender 


Number of 
Gender Participants Mean Rank Sum of Ranks U p 
(N) 
Female 16 15.69 251.00 115.000 842 
Male 15 16.33 245.00 


Table 2 shows no significant differences in students’ 
Dr. Scratch web tool scores according to gender, 
U=115,000, p>,05. Based on this finding, students’ Dr. 
Scratch web tool scores did not differentiate in terms of 
gender. Based on this result, it can be claimed that gender 
does not play a significant role on students’ Dr. Scratch 
web tool scores 


B. Students’ Dr. Scratch web tool scores based on period 
of computer use 


Mann Whitney U test was conducted to examine 
whether students’ Dr. Scratch web tool scores 
significantly differed according to period of computer use. 
Table 3 presents the data obtained from the test. 


Table 3. Investigation of Students’ Dr. Scratch Web Tool Score According to Period of Computer Use 


Period of computer WUnIDESSE 
P Participants Mean Rank Sum of Ranks U p 
use (N) 
Less than | hour 20 16.40 328.00 102.000 5739 
1 —3 hours 11 15.27 168.00 


Table 3 shows that students’ Dr. Scratch web tool 
scores were not statistically significant based on period of 
computer use, U=102,000, p>,05. Therefore, Dr. Stretch 
scores of students’ Scratch projects did not differ 
according to period of computer use. Hence it can be 
argued that period of computer use was not effective on 
students’ Dr. Scratch web tool scores. 


C. Students’ Computational Thinking Levels Based on 
Gender 


Mann Whitney U was conducted to examine whether 
the scores students obtained from the Computational 
Thinking Levels Scale differed according to gender. 
Table 4 presents test results. 


Table 4. Investigation of Student Scores Obtained From Computational Thinking Levels Scale According To Gender 


Number of 
Gender Participants Mean Rank Sum of Ranks U p 
W) 
Female 16 15.31 245.00 109.000 663 
Male 1B) 16.73 251.00 


Table 4 shows no statistically significant differences in 
student scores in Computational Thinking Levels Scale 
based on gender, U=109,000, p>,05. Therefore, scores 
students obtained from Computational Thinking Levels 
Scale did not differentiate according to gender. It can be 
argued that gender did not play an effective role on 
student scores in Computational Thinking Levels Scale. 


D. Students’ Computational Thinking Levels Based on 
Period of Computer Use 


Mann Whitney U was conducted to examine whether 
the scores students obtained from the Computational 
Thinking Levels Scale differed according to period of 
computer use. Table 5 presents test results. 


Table 5. Investigation Of The Scores Students Obtained From Computational Thinking Levels Scale According To Period Of Computer Use 


Period of computer Namber-of 
P Participants Mean Rank Sum of Ranks U p 
use (N) 
Less than | hour 20 16.27 325.50 104.500 5820 
1-3 hours 11 15.50 170.50 


Table 5 presents no statistically significant differences 
in student scores in Computational Thinking Levels Scale 
based on period of computer use, U=149,500, p>,05. It 
can be argued that period of computer use did not play an 
effective role on student scores in Computational 
Thinking Levels Scale. 
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E. Relationship between Students’ Dr. Scratch Web Tool 
Scores and Their Computational Thinking Levels 


Table 6 summarizes the findings regarding the 
relationship between students’ Dr. Scratch web tool 
scores and their computational thinking levels. 
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Table 6. Relationship Between Students’ Dr. Scratch Web Tool Scores And Their Computational Thinking Levels 


Computational Thinking 


Levels 
Spearman's rho Dr. Scratch Web Tool Correlation Coefficient 934" 
Scores Sig. (2-tailed) 000 
N 31 


Table 6 displays a positive relationship between 
student scores obtained from assessing students’ Scratch 
projects via Dr. Scratch Web tool and _ their 
Computational Thinking Levels, 1=,934,  p<,0l1. 
Correlation coefficients between “0.70 and 1” point to a 
high level relationship [30]. Findings suggest that 
increases in students’ programming skills in Scratch will 
cause increases in their computational thinking skills as 
well or increases in students’ Computational Thinking 
Levels will generate increases in their Scratch skills. 


IV. CONCLUSIONS AND DISCUSSION 


Especially in the last decade, studies have been 
conducted at schools about student acquisition in 
programming and computational thinking skills [1,12]. 
Scratch is regarded as a usable tool in teaching 
programming or ensuring that students acquire 
computational thinking skills [12,16]. This study 
compared the scores obtained by 5" graders from Scratch 
projects developed in the framework of Information 
Technologies and Software classes via Dr Scratch web 
tool with the scores obtained from Computational 
Thinking Levels Scale. Results obtained from the current 
study are presented below: 

Scores obtained from assessing Scratch projects via Dr. 
Scratch web tool and students’ Computational Thinking 
Levels do not differ based on gender. In other words, 
gender is not influential on students’ project assessment 
scores. While literature presents extensive proof for the 
impact of gender on students characteristics related to 
computers or programming [31-34], it is also clear from 
the literature that there is a plethora of computing or 
programming features on which gender is not effective 
[35-37]. As a result, it is not possible to make a clear 
distinction in terms of computer use or programming 
related to gender 

Scores obtained from assessing Scratch projects via Dr. 
Scratch web tool and students’ Computational Thinking 
Levels do not differ based on the period of daily 
computer use. It is possible to find evidence in literature 
that student skills regarding computer use or 
programming develop based on their computer or internet 
skills [34,38,39]. However, literature also presents that 
period of computer use does not affect programming 
skills [40]. It can be thought that casual interest on 
computers will naturally develops students’ game 
reflexes and increase their skills in using basic software 
and internet tools. However, training in programming is 
basically training in thinking. Therefore, it should not be 
expected that computer use will develop these skills 
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randomly without a targeted and planned training on 

programming. Result of this study supports this view. 
There is a significantly high relationship between 

students’ Scratch skills and their computational thinking 


skills. In other words, development in students’ 
programming skills in Scratch will cause similar 
increases in their computational thinking skills or 


improvements in their Computational Thinking Levels 
will generate increases in their Scratch skills. Literature 
provides extensive proof that the process of programming 
is not a mechanical process, but a thinking discipline [41]. 
As a matter of fact, ISTE’s [8] computational thinking 
concept is established in this ground. On the other hand, 
Dr. Scratch web tool aims to analyze Scratch projects 
based on sub headings such as logical thinking, problem 
solving, abstraction and analogy/parallelism [20]. ISTE’s 
[8] computational thinking definition provides all these 
sub headings and thinking skills are defined with 
creativity, problem solving, algorithmic thinking, 
cooperation, communication and critical thinking. In this 
context, the high level and significant relationship 
between these two skills can be regarded as a natural 
outcome. 


V. SUGGESTIONS 


Instead of letting students to spend casual time on 
computers in a random manner and providing them with 
basic programming education, students should be 
provided with planned training activities in terms of 
programming and thinking and in this context, thinking 
skills based programming training should be included in 
all phases of education from basic education to university 
grades. 

While developing programs for lessons on codes, 
thinking exercises should be directly provided rather than 
teaching algorithms based on memorization and include 
the general structure of programming languages and 
history of programming 
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